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ABSTRACT 



Techniques for determining an output rate for a bit stream, 
the output rate being determined by applying information 
read from the bit stream to a model of a receiver for the bit 
stream. The techniques are employed to construct a statis- 
tical multiplexer that multiplexes varying bit-rate MPEG-2 
bit streams onto a satellite up-link. Minimum and maximum 
output rates for each MPEG-2 bit stream are determined 
such that neither a queue for the bit stream in the multiplexer 
nor the bit stream's decoder will underflow or overflow. The 
rates with regard to the decoder are determined using timing 
information read from the bit stream and a general model of 
an MPEG-2 decoder. The multiplexer first allocates each bit 
stream its minimum bandwidth and then allocates any 
remaining bandwidth to the bit streams in proportion to the 
difference between the minimum and maximum output rates 
for the bit streams, with no bit stream receiving more than 
its maximum output rate. If there is not enough bandwidth 
to give every bit stream its minimum rate, bit rates are 
allocated according to priorities assigned to the bit streams. 

5 Claims, 10 Drawing Sheets 
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USING A RECEIVER MODEL TO 
MULTIPLEX VARIABLE-RATE BIT 
STREAMS HAVING TIMING CONSTRAINTS 

This application is a divisional of application Ser. No. 5 
08/823,007 filed Mar. 21, 1997, now U.S. Pat. No. 6,052,384 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 1Q 
The invention has to do with the transmission of variable- 
rate bit streams generally and more particularly with the 
efficient time multiplexing of several such bit streams onto 

a transmission medium. 

2. Description of the Prior Art 15 
FIGS. 1-3 

A new problem in data transmission is the transmission of 
data that requires a high band width, is bursty, and has 
temporal constraints. Traditionally, data transmission has 
been done on the public switched networks provided by the 20 
telephone companies and on packet networks. The public 
switched networks are designed for interactive voice 
applications, and so provide relatively low-bandwidth cir- 
cuits that satisfy stringent temporal constraints. The packet 
networks are designed for the transfer of data between 25 
computer systems. The only constraint is that the data 
eventually arrive at its destination. The amount of bandwidth 
available for a transfer depends on the degree of congestion 
in the network. The packet networks thus typically make no 
guarantees whatever about when or even in what order the 30 
data in a burst of data will arrive at its destination. As may 
be seen from the foregoing, neither the telephone network 
nor the packet network is well-adapted to handle high- 
bandwidth bursty data with time constraints. An example of 
such data is digital television which has been compressed 35 
according to the MPEG-2 standard. For details on the 
standard, see Background Information on MPEG-1 and 
MPEG-2 Television Compression, which could be found in 
November 1996 at the URL 

FIG. 1 shows those details of the MPEG-2 standard that 40 
are required for the present discussion. The standard defines 
a encoding scheme for compressing digital representations 
of video. The encoding scheme takes advantage of the fact 
that video images generally have large amounts of spatial 
and temporal redundancy. There is spatial redundancy 45 
because a given video picture has areas where the entire area 
has the same appearance; the larger the areas and the more 
of them there are, the greater amount of spatial redundancy 
in the image. There is temporal redundancy because there is 
often not much change between a given video image and the 50 
ones that precede and follow it in a sequence. The less the 
amount of change between two video images, the greater the 
amount of temporal redundancy. The more spatial redun- 
dancy there is in an image and the more temporal redun- 
dancy there is in the sequence of images to which the image 55 
belongs, the fewer the bits that will be needed to represent 
the image. 

Maximum advantage for the transmission of images 
encoded using the MPEG-2 standard is obtained if the 
images can be transmitted at variable bit rates. The bit rates 60 
can vary because the rate at which a receiving device 
receives images is constant, while the images have varying 
number of bits. A large image therefore requires a higher bit 
rate than a small image, and a sequence of MPEG images 
transmitted at variable bit rates is a variable-rate bit stream 65 
with time constraints. For example, a sequence of images 
that shows a "talking head" will have much more spatial and 
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temporal redundancy than a sequence of images for a 
commercial or MTV song presentation, and the bit rate for 
the images showing the "tauing head" will be far lower than 
the bit rate for the images of the MTV song presentation. 

The MPEG-2 compression scheme represents a sequence 
of video images as a sequence of pictures, each of which 
must be decoded at a specific time. There are three ways in 
which pictures may be compressed. One way is intra-coding, 
in which the compression is done without reference to any 
other picture. This encoding technique reduces spatial 
redundancy but not time redundancy, and the pictures result- 
ing from it are generally larger than those in which the 
encoding reduces both spatial redundancy and temporal 
redundancy. Pictures encoded in this way are called 
I-pictures. A certain number of I-pictures are required in a 
sequence, first, because the initial picture of a sequence is 
necessarily an I-picture, and second, because I-pictures 
permit recovery from transmission errors. 

Time redundancy is reduced by encoding pictures as a set 
of changes from earlier or later pictures or both. In MPEG-2, 
this is done using motion compensated forward and back- 
ward predictions. When a picture uses only forward motion 
compensated prediction, it is called a Predictive-coded 
picture, or P picture . When a picture uses both forward and 
backward motion compensated predictions, it is called a 
Bidirectional predictive -coded picture, or a B picture in 
short. P pictures generally have fewer bits than I pictures and 
B pictures have the smallest number of bits. The number of 
bits required to encode a given sequence of pictures in 
MPEG-2 is thus dependent on the distribution of picture 
coding types mentioned above, as well as the picture content 
itself As will be apparent from the foregoing discussion, the 
sequence of pictures required to encode the images of the 
"talking heads" will have fewer and smaller I pictures and 
smaller B and P pictures than the sequence required for the 
MTV song presentation, and consequently, the MPEG-2 
representation of the images of the talking heads will be 
much smaller than the MPEG-2 representation of the images 
of the MTV sequence. 

The MPEG-2 pictures are being received by a low-cost 
consumer electronics device such as a digital television set 
or a set-top box provided by a CATV service provider. The 
low cost of the device strictly limits the amount of memory 
available to store the MPEG-2 pictures. Moreover, the 
pictures are being used to produce moving images. The 
MPEG-2 pictures must consequently arrive in the receiver in 
the right order and with time intervals between them such 
that the next MPEG-2 picture is available when needed and 
there is room in the memory for the picture which is 
currently being sent. In the art, a memory which has run out 
of data is said to have unde41owed, while a memory which, 
has received more data than it can hold is said to have 
overflowed In the case of underflow, the motion in the TV 
picture must stop until the next MPEG-2 picture arrives, and 
in the case of overflow, the data which did not fit into 
memory is simply lost. 

FIG. 1 is a representation of a digital picture source 103 
and a television 117 that are connected by a channel 114 that 
is carrying a MPEG-2 bit stream representation of a 
sequence of TV images. In system 101, a digital picture 
source 103 generates uncompressed digital representations 
of images 105, which go to variable bit rate encoder 107. 
Encoder 107 encodes the uncompressed digital representa- 
tions to produce variable rate bit stream 109. Variable rate bit 
stream 109 is a sequence of compressed digital pictures 111 
of variable length. As indicated above, when the encoding is 
done according to the MPEG-2 standard, the length of a 
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picture depends on the complexity of the image it represents may be output to medium 207. Each channel 114 in turn has 

and whether it is an I picture, a P picture, or a B picture. a slot 213, and all of the slots taken together make up a time 

Additionally, the length of the picture depends on the slice 211. When medium 207 is carrying channels like 

encoding rate of VBR encoder 107. That rate can be varied. channel 114 that have varying bit rates and time constraints, 

In general, the more bits used to encode a picture, the better 5 slot 213 for each of the channels 114 must output enough 

the picture quality. packets to provide bits at the rate necessary to send the 

Bit stream 109 is transferred via a channel 114 to VBR largest pictures 111 to channel 114 within channel 114 's 

decoder 115, which decodes the compressed digital pictures time, overflow, and underflow constraints. Of course, most 

111 to produce uncompressed digital pictures 105. These in of the time, a channel's slot 213 will be outputting fewer 

turn are provided to television 117. If television 117 is a 10 packets than the maximum to medium 207, and sometimes 

digital television, they will be provided directly; otherwise, may not be carrying any packets at all. Since each slot 213 

there will be another element which converts uncompressed represents a fixed portion of medium 207 *s total bandwidth, 

digital pictures 105 into standard analog television signals any time a slot 213 is not full, a part of medium 207' s 

and then provides those signals to television 117. There may bandwidth is being wasted. 

of course be any number of decoders 115 receiving the 35 In order to avoid wasting the bandwidth of medium 207, 

output of a single encoder 107. a technique is used which ensures that time slice 211 is 

In FIG. 1, channel 114 transfers bit stream 109 as a generally almost full of packets. This technique is termed 

sequence of packets 113. The compressed digital pictures statistical multiplexing. It takes advantage of the fact that at 

111 thus appear in FIG. 1 as varying-length sequences of a given moment of time, each of the channels in a set of 

packets 113. Thus, picture lll(<f) has n packets while picture 20 channels will be carrying bits at a different bit rate, and the 

111(a) has k packets. Included in each picture 111 is timing bandwidth of medium 207 need only be large enough at that 

information 112. liming information 112 contains two kinds moment of time to transmit what the channels are presently 

of information: clock information and time stamps. Clock carrying, not large enough to transmit what all of the 

information is used to synchronize decoder 115 with encoder channels could carry if they were transmitting at the maxi- 

107. The time stamps specify when a picture is to be 25 mum rate. The output of the channels is analyzed statisti- 

decoded and when it is actually to be displayed. The times cally to determine what the actual maximum rate of output 

specified in the time stamps are specified in terms of the for the entire set of channels will be and the bandwidth of 

clock information. As indicated above, VBR decoder con- medium 207 is sized to satisfy that actual peak rate, 

tains a relatively small amount of memory for storing Typically, the bandwidth that is determined in this fashion 

pictures 113 until they are decoded and provided to TV 117. 30 will be far less than is required for multiplexing in the 

This memory is shown at 119 in FIG. 1 and is termed in the manner shown at 210 in FIG. 2. As a result, more channels 

following the decoder's bit buffer. Bit buffer 119 must be at can be sent in a given amount of bandwidth. At the level of 

least large enough to hold the largest possible MPEG-2 slots, what statistical multiplexing requires is a mechanism 

picture. Further, channel 114 must provide the pictures 111 which in effect permits a channel 114 to have a slot in time 

to bit buffer 119 in such fashion that decoder 115 can make 35 slice 211 which varies in length to suit the actual needs of 

them available at the proper times to TV 117 and that bit channel 114 during that time slice 211. Such a time slice 211 

buffer 119 never overflows or overflows. Bit buffer 119 with varying-length slots 215 is shown at 214. 

underflows if not all of the bits in a picture 111 have arrived One method of statistically multiplexing bit streams is 

in bit buffer 119 by the time specified in the picture's time disclosed in Rao, U.S. Pat. No. 5,506,844, Method for 

stamp for decoder 115 to begin decoding the picture 111. 40 Configuring a Statistical Multiplexer to Dynamically Ailo- 

Providing pictures 111 to VBR decoder 115 in the proper cate Communication Channel Bandwidth, issued Apr. 9, 

order and at the proper times is made more complicated by 1996. FIG. 3 is an overview of the method disclosed in the 

the fact that a number of channels 114 may share a single Rao application. System 301 includes a set of encoders 

very high bandwidth data link. For example, a CATV 302(0 . . . n) which encode a set of bit streams 105(0 . . . n). 

provider may use a satellite link to provide a large number 45 During a given period of time, termed herein a window, each 

of TV programs from a central location to a number of encoder 302(0 encodes at a constant bit rate; however, the 

CATV network head ends, from which they are transmitted bit rate may be changed at the beginning of the window. The 

via coaxial or fiber optic cable to individual subscribers or output of an encoder 302(0 is thus a bit stream 108(0 having 

may even use the satellite link to provide the TV programs a pieccwise-constant bit rate. The bit streams 108(0 . , . n) 

directly to the subscribers. When a number of channels share 50 are input to multiplexer 303, which multiplexes them onto 

a medium such as a satellite link, the medium is said to be medium 207. 

multiplexed among the channels. Multiplexer 303 maximizes the use of medium 207 by 

FIG. 2 shows such a multiplexed medium. A number of adjusting the bit rates of encoders 302(0 ... n). As 

channels 114(0) through 114(n) which are carrying packets mentioned above, there is a relationship between bit rate and 

containing bits from variable rate bit streams 109(0 . . . n) 55 picture quality. Generally, the higher the bit rate, the better 

are received in multiplexer 203, which processes the packets the picture quality. Consequently, in adjusting the bit rates of 

as required to multiplex them onto high bandwidth medium encoders 302(0 . . . n), multiplexer 303 must be aware of the 

207. The packets then go via medium 207 to demultiplexer current picture quality of each bit stream and must adjust the 

209, which separates the packets into the packet streams for bit rates not only to maximize the use of medium 207, but 

the individual channels 114(0... n). A simple way of sharing 60 also to maximize the picture quality of each of the bit 

a high bandwidth medium among a number of channels that streams 108(0- 

are carrying digital data is to repeatedly give each individual As Mux 303 operates, it receives information from each 

channel 114 access to the high bandwidth medium for a short encoder 107(0 that indicates the picture distortion rate for 

period of time, termed herein a slot. encoder 107(i) , s current encoding rate (DIF 311(0) 

One way of doing this is shown at 210 in FIG. 2. The short 65 keeps track of the fullness of encoding buffer 307(0 m 

period of time appears at 210 as a slot 213; during a slot 213, encoder 107(0, as shown by arrow EBF 309(0- Encoding 

a fixed number of packets 113 belonging to a channel 114 buffer 307(0 holds bit stream 105(0 w hil e it is being 
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encoded, and encoder 107(i) must encode at a rate such that remaining bandwith for a bit stream is done proportionally 

encoding buffer 301(f) neither overflows nor underflows. to the difference between the maximum and minimum rates 

Multiplexer 303 determines from the current distortion rates for the bit stream. 

of the encoders 107 which encoders need to encode at a j n another aspect of the invention, the multiplexer further 

higher bit rate and which can encode at a lower bit rate and s priority information about the bit streams in adjusting 

at the beginning of a window, it adjusts the rate of each the rates of the bit streams when there is not enough 

encoder 107, as indicated by the arrows BRCTL 305(0 . . . bandwidth for all of the bit streams to receive the minimum 

n), to maximize the picture quality for all of the encoders amount. Each bit stream is either apriority one, priority two, 

107 while maximizing the degree to which medium 207's or priority three bit stream, with priority one bit streams 

bandwidth is used. When multiplexer 303 reduces or 10 being required to meet the time constraints set in the bit 

increases an encoder 107(t)'s bit rate, it also reduces or stream, priority two bit streams being required only not to 

increases the size of EBUF 307(0 m the encoder. underflow, and priority three bit streams being permitted to 

While the statistical multiplexer of Rao does maximize underflow. Where bandwidth is necessary for priority one or 

the degree to which medium 207*s bandwidth is used, it has priority two bit streams, it is taken from priority two and/or 

a number of shortcomings. Perhaps the most important of 15 priority three bit streams. 

these is that it adjusts the multiplexing by changing picture Stffl anothef of me invemion fc a mu i tipleX er for 

quality. The system thus cannot guarantee any user a given MpEG _ 2 bit strcams ^ statistical mu i t i p lcxcr multiplexes 

quality of picture. ... , a plurality of MPEG-2 bit streams and constant bit rate audio 

Another shortcoming is that it requires encoders that bft strcams 0flt0 afl mcdium Induded m mis 

encode digital images as piecewise-constant bit streams. 20 of ^ invention are a receiver model for MPEG _ 2 bit 

Such bit streams have a lower degree of compression than strcams and al rithms forusi thc modcl t0 dctcrminc thc 

variable-rate bit streams; further, the encoding rate and t bU fate for a Mp£G . 2 Wt stream 

therefore the quality of the picture changes at the beginning . 

of each window; with sequences of fast changing images, u ™ ese ™ d other ^ cts Z *]* ° ""T,"? 1 

this will produce coding artifacts in the pictures. 25 become a PP» rent t0 those ™} h \ *f to ^ . h « 

Still another is that the multiplexing requires feedback mven ^° n P erta "* u P on P"" 5 * 1 . of ^ f°U°*™S Detailed 

from multiplexer 303 to encoders 302(0 . . . n). One Descnption and Drawing, wherein: 

consequence of this fact is that multiplexer 303 will not BRIEF DESCRIPTION OF THE DRAWINGS 

work with pre -stored sequences of pictures 111; another is 

that in order to use information like encoder buffer fullness 30 FIG. 1 is a diagram showing how digital television 

309 and distortion information 311 to allocate bandwidth in pictures are encoded, transmitted, and decoded; 

medium 207, multiplexer 303 must take into account the FIG. 2 is a diagram showing multiplexing of variable -rate 

inner workings of encoder 107. A third is that there must be bit slreams onto a mgh band width me dium; 

a high-speed connection between multiplexer 303 and each CT ~ - . „ « - n • „♦ „*„*„*:„..i i*j 

• 4 . 4 . . , . f r- II FIG. 3 is a block diagram of a prior- art statistical multi- 
encoder 302 to exchange the control information. Finally, 35 . & r 

the bitrate switching of the encoders and the multiplexer is p ' 

difficult to implement, particularly if it is necessary to FIG. 4 is a block diagram of a statistical multiplexer 

support video inputs having different frame rates. which implements a preferred embodiment of the invention; 

It is an object of the invention disclosed herein to over- FIG. 5 is a more detailed block diagram of a part of the 

come these shortcomings and thereby to provide an 40 statistical multiplexer of FIG, 4; 

improved statistical multiplexer. FIG. 6 is pseudo-code for the algorithm used to determine 

SUMMARY OF THE INVENTION ^ bit rate of a channel in ^ P referred embodiment; 

m . . , , FIG. 7 is a flow chart for the algorithm used to allocate the 

The invention is embodied in a multiplexer that over- ^ bU fate of medium m among the channds; 

comes the foregoing problems of the prior art by determm- 45 ™^ « . . , L , , Jt c . ..... . 

ing the rate at which each bit stream is to be output to the F °: 8 * a ««*ptual block diagram of the statistical 

medium in accordance with rate information obtained by mu lp exe *' 

applying a model of a receiver for each of the bit streams to p I G - 9 is - a high-level block diagram of an encoding 

information contained in the bit stream itself. The rate for system which includes an implementation of the statistical 

each bit stream is determined such that the receiver for the 50 multiplexer; 

bit stream will neither underflow nor overflow and the FIG. 10 is a more detailed view of the implementation of 

multiplexer's use of the medium's bandwidth is maximized. the statistical multiplexer; ' 

Advantages of the multiplexer of the invention include the FIG. 11 is a detailed view of a channel input block in the 

fact that all of the information needed to determine the rates statistical multiplexer of FIG, 10; and 

is available within the multiplexer itself, the fact that the 55 FIG. 12 is a flowchart of the minimal bitrate algorithm, 

technique will work with any kind of bit stream for which it ...... , 

is possible to make a receiver model that is able to determine A . ^ r ff crcncc nun > beni m * c ^ mn ^ h ,. avc at least th / ee 

an output rate on the basis of information read from the bit ™? two nghta«* *&* •» reference numbers 

stream, and the fact that adjustment of the output rates does a ^ < hc l ° th K c }f of tho f d f * a K re * e 

t n(T t ~p u-t number of the figure in which the item identified by the 

not affect the contents or the bit stream. 60 ^ . & « ^ • . 

T tl _ t . . - c i. , reference number first appears. For example, an item with 

In another aspect, the rate information for each bit stream f . c r \ „ f ~ J 

. , , r . . a • L1 A , . reference number 203 first appears in FIG. 2. 
is provided as a minimum allowable rate and a maximum 

allowable rate and the multiplexer allocates bandwidth to the DETAILED DESCRIPTION 
bit streams by first giving each bit stream its minimum 

allowable rate and then allocating any remaining bandwidth 65 The following Detailed Description will first present an 

to the bit streams up to the maximum rate for each bit overview of the preferred embodiment, will then provide a 

stream. In the preferred embodiment, allocation of the description of the hardware in which the preferred embodi- 
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merit is implemented, and will finally provide a detailed 
description of the algorithms used to allocate bandwidth in 
the preferred embodiment. 
Conceptual Overview: FIG. 8 

FIG. 8 presents a conceptual overview of a statistical 
multiplexer 801 which incorporates the principles of the 
invention. A number n of variable-rate bit streams 109 are 
received in receiver 803, which provides them to bandwidth 
portion controller 805. Bandwidth portion controller 805 
dynamically determines what portion of the bandwidth of 
medium 801 that each bit stream 109(f) is to receive and 
provides a corresponding portion 815(0 of the bit stream to 
transmitter 817, which outputs the portions 815(0 ... n) it 
receives of each bit stream 109(0 . . . n) onto medium 207. 

Bandwidth portion controller 805 has a number of sub- 
components. There is a transmission controller 807(0 for 
each bit stream 109(0- Each transmission controller 807(0 
contains a bit stream analyzer 809(0 and a receiver model 
811(f)- Bit stream analyzer 809(0 collects information from 
bit stream 109(0 and applies receiver model 811(0 to the 
collected information to determine what rate is required by 
the condition of the receiving device. In the case of a 
MPEG-2 bit stream, the receiving device is a decoder 115(0; 
and for such a decoder, the required rate can be determined 
from the time stamps and the sizes of the pictures making up 
bit stream 109(0- Transmission controller 807(0 applies 
receiver model 811(0 to this information to determine rate 
information 812(0- Bandwidth allocator 813 receives rate 
information 812(0 . . . n) and uses this information to 
allocate the portion of the bandwidth of medium 207 that 
each bit stream 109(0 * s t° receive. Having done this for 
each bit stream 109(0 , . . n), it provides a bit stream portion 
815(0 tn at corresponds to the allocated bandwidth to trans- 
mitter 817. 

It is worth noting here that all of the information required 
by the above technique for allocating bandwidth can be 
obtained by applying the receiver models 811 to the infor- 
mation received from the bit streams 109 and that informa- 
tion need only be exchanged between bandwidth allocator 
813 and transmission controllers 807. There is no need 
whatever to receive information from or provide information 
to the encoders 107. Put another way, all of the information 
needed to allocate the bandwidth is available within statis- 
tical multiplexer 801 itself. 

It is also worth noting that the technique of using a model 
of a receiver to control the rate at which a bit stream is output 
to a receiver may be applied in other situations, For example, 
a receiver model could be used to control the rate at which 
a MPEG-2 encoder encoded data. 
Overview of a Preferred Embodiment: FIG. 4 

FIG. 4 provides an overview of a statistical multiplexer 
401 for MPEG-2 bit streams which is implemented accord- 
ing to the principles of the invention. The main components 
of multiplexer 401 are packet collection controller 403, a 
transmission controller 407(0 for each variable-rate bit 
stream 109(0, a packet delivery controller 419, and a 
modulator 423, which receives the output of packet delivery 
controller 419 and outputs it in the proper form for trans- 
mission medium 207. Packet collection controller 403 col- 
lects packets from variable-rate bit streams 109(0 . . . n) and 
distributes the packets that carry a given bit stream 109(0 t0 
the bitream's corresponding transmission controller 407(0- 
In the preferred embodiment, the packets for all of the bit 
streams 109(0 . . . n) are output to bus 402. Each packet 
contains an indication of which bit stream it belongs to, and 
packet collection controller responds to the indication con- 
tained in a packet by routing it to the proper transmission 
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controller 407(0 H should be noted here that the packets in 
each bit stream 109(0 arrive in transmission controller 
407(0 in the order in which they were sent by encoder 
107(0. 

Transmission controller 407(0 determines the rate at 
which packets from its corresponding bit stream 109(0 is 
output to medium 207. The actual rate determination is made 
by transmission rate controller 413, which at a minimum, 
bases its determination on the following information: 
for at least a current picture 111 in bit stream 109(0, the 
timing information 112 and the size of the current 
picture. 

a Video Buffer Verifier (VBV) model 415(0, which is a 
model of a hypothetical bit buffer 119(0- 
VBV model 415(0 uses the timing information and picture 
size information to determine a range of rates at which bit 
stream 109(0 must be provided to the decoder's bit buffer 
119(0 if bit buffer 119(0 * s to neither overflow nor under- 
flow. Transmission rate controller 413(0 provides the rate 
information to packet delivery controller 419, which uses 
the information from all of the transmission controllers 407 
to determine during each time slice how the bandwidth of 
transmission medium 207 should be allocated among the bit 
streams 109 during the next time slice. The more packets a 
bit stream 109(0 needs to output during a time slice, the 
more bandwidth it receives for that time slice. 

Continuing in more detail transmission controller 407 
obtains the timing and picture size information by means of 
bit stream analyzer 409, which reads bit stream 109(0 ^ ^ 
enters transmission controller 407 and recovers the timing 
information 114 and the picture size 411 from bit stream 
109(0- Bit stream analyzer 409 can do so because the 
MPEG-2 standard requires that the beginning of each picture 
111 be marked and that the timing information 114 occupy 
predetermined locations in each picture 111. As previously 
explained, timing information 114 for each picture 111 
includes a clock value and a decoding time stamp. Trans- 
mission controller 407(0 and later decoder 115(0 u$e the 
clock value to synchronize themselves with encoder 107(0- 
The timing information is found in the header of the PES 
packet that encapsulates the compressed video data. The 
information is contained in the PTS and DTS time stamp 
parameters of the PES header. The MPEG-2 standard 
requires that a time stamp be sent at least every 700 msec. 
If a compressed picture is not explicitly sent with a com- 
pressed picture, then the decoding time can be determined 
from parameters in the Sequence and Picture headers. For 
details, see Annex C of ISO/IEC 13818-1. Bit stream 
analyzer 409 determines the size of a picture simply by 
counting the bits (or packets) from the beginning of one 
picture to the beginning of the next picture. 

The timing information and the picture size are used in 
VBV model 415(t). VBV model 415(0 requires the timing 
information and picture size information for each picture in 
bit stream 109(0 fr° m the time the picture enters multiplexer 
401 until the time the picture is decoded in decoder 115(0- 
DTS buffer 414 must be large enough to hold the timing 
information for all of the pictures required for the model. It 
should be noted here that VBV model 415(0 's behavior is 
defined solely by the semantics of the MPEG-2 standard, not 
by any concrete bit buffer 119(0- Any bit buffer for a 
working MPEG-2 decoder must be able to provide the 
decoder with the complete next picture at the time indicated 
by the picture's timing information; that means that the bit 
buffer 119(0 f° r anv working MPEG-2 decoder must be at 
a minimum large enough for the largest possible MPEG-2 
picture. Given this minimum buffer size, the timing infor- 
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mation for the pictures, and the sizes of the individual as required to simultaneously satisfy the ranges of rates and 
pictures, VBV model 415(0 can determine a rate of output priorities provided by TRC 413 for each transmission con- 
fer bit stream 109(0 which will guarantee for bit buffers trolier 407(0 and maximize the number of packets 113 
19(0 of any working MPEG-2 decoder that each picture output during time slice 211. In the preferred embodiment, 
arrives in the bit buffer 19(0 before the time it is to be 5 controller 419 has two components, central bit rate control- 
decoded and that there win be no overflow of bit buffer ler 501 which is a processor that analyzes the information 
119(0 received from each of the transmission rate controllers 413 

Details of Transmission Controller 407 and Packet Delivery ™ ^I****™™ ^"""J* P ac * e te fr ? m ^ h bl } 

Controller 419* FIG 5 stream 109(0 are t0 be output in the next time slice 211, and 

e l *i_ j . -i c r j u j* * f switch 511, which takes the number of packets 113 permitted 

FIG. 5 shows the details of a preferred embodimen of 10 Mt ^ fof eacfa bh stfeam J duri ^ ^ 

^ansmission controller 407 and packet delivery controller ^ m Swii ^ sn ^ lamented so as to deliver packets 

419. The figure shows three of the n transmission from eacfa 5Q9(0 such that ^ ^ are eyenl 

controllers, namely transmission controllers 407(i . . . and distributed across time slice 211. Implementing switch 511 

the two major components of packet delivery controller 419, in this way rcduccs ±e burstincss of thc strcam of packcts 

namely central bit rate controller 501 and switch 511. 15 io9(0 to decoder 115(0 ^d thereby reduces the amount of 

Beginning with transmission controller 407(0, in addition to transport packet buffer needed in decoder 115. Such imple- 

transmission rate controller 413, analyzer 409, and VBV mentations of switch 511 are well-known in the art. 

model 415, transmission controller 409 includes statistical An important advantage of multiplexer 401, or indeed of 

multiplexer buffer (SMB) 507, a meter 505 for buffer 507, any statistical multiplexer built according to the principles of 

and throttle 509. 20 the invention is that the multiplexer can simultaneously 

SMB 507(0 is a first-in-first-out pipe buffer which holds multiplex both constant-rate and variable-rate bit streams 

the bits of bit stream 109(0 while they are in transmission onto medium 207. The reason for this is that as far as 

control 407(0- in lnc preferred embodiment, SMB 507(0 statistical multiplexer 401 is concerned, a constant- rate bit 

receives pictures 111 in bursts that contain all or almost all stream is simply a degenerate case: it is a varying-rate bit 

of the bits in the picture, depends on the picture size and 25 stream whose rate never varies. Thus, with a constant-rate 

maximal bit rate specified by the encoder. Such bursts are bit stream, TRC 413(0 always returns the same rate infor- 

termed herein picture pulses, and the time period represented mation 417(0 t0 packet delivery controller 419. 

by such a picture pulse is denoted as T^, which is the inverse Hardware Implementation of a Preferred Embodiment: 

of video frame rate. For example, Tp= 1/29.97-33 ms for FIGS. 9-11 

NTSC video coding. As previously stated, packet delivery 30 A presently-preferred embodiment of the invention is 

controller 419 provides packets in time slices 211, The implemented as a modification of the PowerVu satellite 

length of time of one of these slices is denoted herein as T c . up-link system manufactured by Scientific- Atlanta, Inc. 

In a preferred embodiment, T c , is 10 ms. (PowerVu is a trademark of Scientific -Atlanta). FIG. 9 is a 

SMB 507(0 musl °f course be large enough to be able to high-level block diagram of the Power- Vu up-link system as 

accept picture pulses of any size during the time it takes to 35 modified to implement the invention. System 901 includes a 

read out the largest expected picture pulse. SMB 507(0 sc * °f encoders 911(0 . . . n). Each encoder 911(0 encodes 

further must be emptied at a rate that ensures that it cannot a video input 903(0 aQ d an audio input 905(0; tne video 

overflow, since that would result in the loss of bits from bit input is encoded at a constant or variable bit rate and the 

stream 109(0- ^ a l so should not underflow, since that would audio input is encoded at a constant bit rate. Each encoder 

result in the insertion of null packets in the bit stream, 40 911(0 has an output 913(0 which carries the encoded video 

resulting in the waste of a portion of the multiplexed and audio. In the PowerVu system as modified, the outputs 

medium. Meter 505 monitors the fullness of SMB 507(0 and 913(0 ... d) go to statistical multiplexer 915, which outputs 

provides information concerning the degree of fullness to a constant bit-rate stream 917 to a modulator for transmis- 

TRC 413(0- 413(0 men uses this information to vary sion to a communications satellite. At a high level, operation 

the range of bit rates that it provides to packet delivery 45 of all of the components of system 901 is supervised and 

controller 419 as required to keep SMB 507(0 fr° m over " controlled by control processor 907, which communicates 

flowing or underflowing. In other embodiments, the degree with the other components by means of Ethernet protocol 

of fullness from meter 505 can also be fed back to encoder 909 (Ethernet is a registered trademark of Xerox 

107(0 and used there to increase or decrease the encoding Corporation). In the presently-preferred embodiment, statis- 

rate. It should be noted here that feeding back the degree of 50 tical multiplexer 915 is implemented as a separate chassis 

fullness to encoder 107(0 does not create any dependencies which need only be coupled to the rest of the PowerVu 

between statistical multiplexer 401 and a given type of system by encoded data inputs 913(0 . . . n), Ethernet 

encoder 107. Throttle 509, finally, is set by TRC 413 on the protocol 909, and output 917. 

basis of information 418(0 mat ^ nas received from packet FIG. 10 shows the preferred embodiment of statistical 

delivery controller 419 to indicate the number of packets 113 55 multiplexer 915 in more detail. Multiplexer 915 receives its 

that bit stream 109(0 is to provide to medium 207 *in time inputs of encoded video and audio from optical fibers. Each 

slice 211. SWIF receiver 1001(0 receives input from a single optical 

In determining the range, TRC 413 sets the minimum rate fiber and there are receivers 1001(0 . . . n) corresponding to 

for a given time slice 211 to the maximum of the rate encoders 911(0 . . . n). Each receiver converts the informa- 

required to keep SMB 507 from overflowing and the rate 60 tion from photons to digital electronic form and outputs it 

required to keep VBV model 415(0 from underflowing and via PCR MOD 1005(0 to channel input block 1009(0- PCR 

the maximum rate for the time slice to the minimum of the MOD 1005(0 corrects the clock information in the encoded 

rate required to keep SMB 507 from underflowing and the video and audio to compensate for any delays in the encod- 

rate required to keep VBV model 415(0 fr° m overflowing. ing process. The synchronization information needed to do 

Continuing with packet delivery controller 419, packet 65 this is provided by MSYNC lock up 1003. 

delivery controller 419 allocates the packets 113 that can be Channel Input 1009(0 is ^ implementation of transmis- 

ou tput during the time slice 211 T c to bit streams 109(0 ... n) sion controller 407(0- Channel input 1009(0 employs a 
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software implementation of VBV model 415 to dynamically 
determine a current rate at which the input from receiver 
1001(f) must be output to multiplexed output stream 917 and 
provides that rate information to central bit rate controller 
1007, which in turn actually allocates a specific rate to 
channel input block 1009(f). Channel input block 1009(f) 
then outputs bits in its bit stream to bus 1011 at that rate. The 
combined outputs of blocks 1009(0 . . . n) then go via 
multiplexed output 1013, PCR MOD 1016, and SWIF 
transmitter 1017 to output 917. PCR MOD 1016 modifies 
the clock information in the encoded video again to deal 
with the time spent in channel input block 1009(f) and 
outputs the bit stream to SWIF transmitter 1017, which 
converts the bit stream to a photonic representation and 
outputs it to an optical fiber. Communication processor 1015 
provides high level control to central bitrate controller 1007 
and also serves as the interface to PCC 907, a control 
console, and a system which broadcasts status information. 
Communications processor 1015 also receives MPEG-2 
service information tables from PCC 907 and provides them 
to service information table insertion 1018, which inserts 
them into the bit streams. 

A presently-preferred embodiment of a single channel 
input block 1009(f) is shown in more detail in FIG. 11. The 
main components are packet director 1101, which detects 
audio packets, video packets, and headers and routes them to 
different components of input block 1009(f), storage 1115 
for the headers, storage 1117 for a FIFO (queue) to hold 
video packets from the time they are received in input block 
1009(i) until they are output to data bus 1011, and a bypass 
FIFO 1119 which holds the constant bit rate audio packets 
while they are in input block 1009(f). Output from FIFO 
1117 is controlled by throttle 1032 under control of throttle 
counter 1123, which specifies the number of packets to be 
output from FIFO 1117 during a given time slot. Output 
from FIFO 1127 is controlled by throttle 1129, which is 
controlled by throttle counter 1123, Throttle counter 1123 is 
set by channel controller 1113 in response to the rate selected 
by central bit rate controller 1007. Throttle counter 1127, 
which is for a constant-rate bit stream and does not depend 
on VBV model 415(f), is set directly by central bit rate 
controller 1007. 

Operation of input block 1009(f) is as would be expected. 
Serial bit stream 1001(f) from SWIF receiver 1001(f) is 
modified by PCR 1005(f) and is output to packet director 
1101, which detects packets, determines their types, and 
outputs them to the various components of channel input 
block 1009(f). Packet director 1101 further provides a start 
of picture interrupt 1103 to channel controller 1113 to 
indicate that a new picture is being received in SMB FIFO 
1117. Channel controller 1113 responds to interrupt 1103 by 
using picture size information obtained from picture counter 
1107, header information stored in header storage 1115, and 
information about the amount of space left in SMB FIFO 
1117 in the VBV model 415(f) to obtain maximum and 
minimum rates at which data must be output from SMB 
FIFO 1117 to avoid overflow or underflow in SMB FIFO 
1117 and overflow or underflow in VBV model 415(f). 
Channel controller 1113 outputs these rates via 1121 to 
central bitrate controller 1007, which selects a rate for the 
next time slice on the basis of the information from channel 
controller 1113, the current output requirements of all of the 
other channel controllers 113, and the total capacity of the 
output stream. Central bitrate controller 1007 returns the 
selected rate to channel controller 1113, which sets throttle 
counter 1132 accordingly. Throttle counter 1132 then deter- 
mines how many bits are actually output by throttle 1125 
during the next time slice. 
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As shown in FIG. 11, packet director 1101 is implemented 
by means of gate arrays and a dual port RAM memory. 
Counters 1107 and 1123 are also implemented using gate 
arrays and channel controller 1113 is a digital signal pro- 

5 cessor. Central bitrate controller 1007 is implemented using 
a microprocessor with a support IC 
Detailed Description of Algorithms used to Compute the 
Output Rate for a Bit stream 109(f) from Statistical Multi- 
plexer 401: FIGS. 6,7, and 12 

10 As indicated above, the maximum rate R^ at which a 
transmission controller 407(f) may output packets 113 to 
medium 207 is determined by the need to keep SMB buffer 
507(f) from underflowing and bit buffer 119(f) from over- 
flowing. The minimum rate R min is determined by the need 

15 to keep SMB buffer 507(f) from overflowing and bit buffer 
119(f) from underflowing. Bit buffer 119(f) will not under- 
flow if all packets belonging to the picture currently being 
sent arrive in bit buffer 119(f) before the time indicated in the 
DTS stamp for the picture. 

20 There are thus two maximum rates and two minimum 
rates that need to be taken into account in determining R max 
andR„^: 

R maxl is the maximum rate at which bit buffer 119(0 m anv 
MPEG-2 decoder that conforms to the standard will not 
25 overflow; 

R maA 2 is the maximum rate at which SMB 507(f) will not 
underflow; 

R minl is the minimum rate at which bit buffer 119(f) will not 
underflow; and 

30 R min2 is tne minimum rate at which SMB 507(f) will not 
overflow. 

R max and R m ^ are determined from the above four maxima 

and minima as follows: 
R«** is the minimum of R max and R^. 

35 Rmin ^ the maximum of R minl and R mirt2 . 

What is needed to compute R m£nl and R maxl is a VBV model 
415(f) that models the fullness and emptiness of bit buffer 
119(f); what is needed to compute R min2 and R max2 is a 
measure of the fullness and emptiness of SMB buffer 507(f). 

40 The model for the fullness of bit buffer 119(f) is termed 
herein VBV fullness and the model for the emptiness of bit 
buffer 119(f) is termed herein VBV emptiness. The algo- 
rithms for measuring VBV emptiness and SMB buffer 
emptiness and fiullness are simple and will be dealt with 

45 first; the algorithm for measuring VBV fullness is substan- 
tially more complex. 

in the case of SMB 507(f), the measure of SMB 
emptiness, E^^, is the amount of free space remaining in 
SMB 507(f). For a given time slice T c 211(m), it is defined 

50 as follows: 

E SMB -SMB_SlZE-F SMB (m) 

where F SMB is the actual SMB fullness measured by the 
Meter 505. Since there is a maximum size for MPEG-2 

55 pictures, termed herein VBV_SIZE, the way to prevent 
SMB 507(f) from overflowing is to guarantee that there is 
always an empty space in SMB 507(f) that is larger than or 
equal to VBV_SIZE. If the free space becomes less than 
that, the minimum rate with regard to SMB 507(f), R min2 , 

60 must be increased in the next lime slice T c (m+1) according 
to the algorithm below; 
if(E^<VBV_SIZE){ 

R m/ „ 2 (m + l>(VBV_SIZE-E Af ^m))/T c ; 

65 > 

R maxfZ is computed as follows: 
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Continuing with the determination of R mi>ll for the next T c 
from VBV model 415(0, the rate can be found from the 
information in VBV model 415(f) concerning the pictures 
111 in SMB 507(f). The rule is simply this: the minimum bit 
rate must be such that the picture currently being output is 
completely output from SMB 507(0 before the time indi- 
cated by its DTS time stamp. One implementation is 

Rmin i (m+l)«pic_residual_bits (q)I(DTS_V maJI -r); 

Here, pic__residuaL_bits is the number of bits of the picture 
111 remaining in SMB 507(f), q is the index of the picture 
currently being transmitted from SMB 507(f) and q+1, q+2, 
... are the indexes of the following pictures, DTS_V ma;e is 
the time stamp with the most recent time in VBV model 
415(f), and t is the actual time determined by the synchro- 
nization time value in the bit strean. 

The above algorithm guarantees that all bits belonging to 
the picture 111 which is currently being delivered to bit 
buffer 119(f) will have been delivered before the decoding 
time DTS_V mar arrives. This algorithm may leave only one 
coded picture in the decoder's bit buffer for decoding. While 
this picture could be decoded correctly, a high bit rate will 
be necessary to deliver the next picture on time such that all 
the bits belong to the next picture, p+1, will be available for 
decoding at the next decoding time instance. This require- 
ment will result in a high bitrate requirement for next Tc 
period and will introduce congestion in the delivery media 
at the next Tc period. A better algorithm is one that guar- 
antees at least two pictures (or more, as long as VBV model 
415(f) does not indicate an overflow) in bit buffer 119(f), 
such as the following: 

^^ 1 (M+l)=pic_rcsidual_bits(^)/(Dr5(?-l)-0 

In this scheme, the minimal bitrate calculation is slightly 
changed by using the second largest value of DTS in bit 
buffer (119(f), DTS (q-1). That is the time stamp for the 
picture 111 preceding the last picture 111 to be sent to bit 
buffer 119(f). This scheme guarantees that the picture p has 
already be delivered to decoder 115(0 at t-DTS (q-1). Of 
course, it is even better to set up the minimal bit rate so that 
the number of coded pictures in bit buffer 119(0 * s usually 
more than 2. 

Determining VBV Fullness: FIG. 6 

When there is no need to prevent overflow of SMB 507(0, 
the maximum bitrate of bit stream 109(0 * s determined from 
the VBV fullness indicated by VBV model 415(0- 

The greater the VBV fullness indicated by the model, the 
less the maximum bitrate. At the beginning of the operation 
of model 415(0, SMB 507(0 is empty and VBV fullness 
indicates that model 415(0 is empty. As soon as bits appear 
in SMB 507(0, central bitrate controller 501 begins output- 
ting them at a predetermined initial rate, for instance, the 
average rate for such variable-rate bit streams. As bits are 
received in SMB 507(0 an ^ output to medium 207, the 
picture information in VBV model 415(0 k updated each 
time slice. The newly updated information is used to com- 
pute VBV fullness for the next time slice and the VBV 
fullness is used in turn to determine the maximum bit rate 
Rmaxi al which bits will be output on bit stream 109(0 for 
the next time period. The computation is the following: 

^-xi('"+l)-CVBV_SIZE_J : '^(myr c 

where F vi(V is the VBV fuilness measure provided by VBV 
model 415(0 anc * En and in+1 are the current and next time 
slices T c 211. 

In the preferred embodiment, the computation of F vM , (m) 
is governed by the following considerations: 
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The calculation requires a computation of the number of 
pictures 111 are currently contained in VBV model 
415(0. 

The calculation requires a knowledge of how many bits of 
the picture 111 which is currently being transmitted 
from SMB 507(0 presently remain in SMB 507(0- 
The data items used to compute F^ (m) in the preferred 
embodiment include the following: 

a. VBV_SIZE, that is, the maximum size of a MPEG-2 
picture. 

b. The absolute maximum bit rate R,^ which packet 
delivery controller 419 can provide to bit stream 109(0- 

c. The current time, t, recovered from the clock time 
information of bit stream 109(0- 

d. Data items for each picture presently in SMB 507(f): 
packet_cnt, the number of packets 113 in the picture, 
DTS, the time stamp for the picture, q, the index for 
DTS and packet_cnt for the picture currently leaving 
SMB 507(0, and r, the index for those values for the 
oldest picture for which there is still information in 
model 415(0- 

e. Status data items in VBV model 415(0 that are updated 
every T c 211: pic_cnt_VBV, the number of pictures 
111 which are presently represented in VBV model 
415(0; pic_residual__bit(q), the number of bits of 
picture 111 q that is currently being transmitted to 
decoder 115(f) that remain in SMB 507(0; DTS_V max , 
the time stamp with the most recent time stamp value 
that is presently in VBV model 415(0; and F^ itself. 

As soon as SMB 507(0 begins receiving bit stream 109(i), 
packet delivery controller 419 sets throtde 509(0 t0 tne 
initial rate provided by central bit rate controller 509. As 
packets are read from SMB 507(0 at that rate, transmission 
rate controller 413(f) updates DTS_V m<rc pic_cnt_VBV, 
F^, and pic_residual_bits (q) as required by the transmis- 
sion of pictures from SMB 507(0 to decoder 115(0 and by 
the addition of bits to SMB 507(f). The algorithm 601 used 
to do this in a preferred embodiment is shown in FIG. 6. 
40 Section 603 of algorithm 601 shows how the parameters are 
initialized at the time the first picture arrives in SMB 507(0. 
Execution of loop 604 begins when the first bits of the 
picture arrive in SMB 507(0- As shown at 605, the loop is 
executed once every T c 211. At the beginning of each 
45 execution of loop 604, pic_residual_bits is decremented by 
the number of bits that were sent at the rate R (m) previously 
determined for the current T c 211 by central bitrate control- 
ler 501. 

At 607, F vfcv is computed. There arc two cases. In the first 
case, shown at 609, the time stamp DTS for the current 
picture r in VBV model 415(0 indicates a time that is after 
the current time t for bit stream 109(0, so decoding of the 
picture r cannot yet have begun. Consequently, the bits that 
were sent during the last T c 211 are simply added to the bits 
that are already in VBV model 415(0 and V vbv is incre- 
mented by that amount. If the comparison of t and DTS (r) 
indicates that decoder 115(0 has already begun decoding the 
picture r, the second case, shown at 611, is executed. 
pic_cnt_VBV is decremented to indicate that one less 
60 picture is now represented in VBV model 415(0 an d ^vb» ls 
adjusted by the difference between the number of bits sent 
to decoder 115(0 in the last T c 211 and the total number of 
bits in the picture that is no longer represented in VBV 
model 415(f). After picture r is removed from VBV model, 
65 415(0, tne index r is incremented by 1. 

Block of code 613 deals with the updating that has to be 
done when a picture q has been completely read from SMB 
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507(f). When that is the case, pic_residual__bits. will have 
a value that is less than or equal to 0. The first updating that 
has to be done is shown at 615. The time stamp DTS for the 
picture 111 that was just sent is now the maximum DTS in 
Bit buffer 119(0, s° DTS_ V m(W is updated with DTS (q). A 
picture q has also been added to the pictures represented in 

VBV model 415(0, so pic_cnt VBV is incremented 

accordingly. The second updating is at 617. The new current 
picture is the next picture in SMB 507(0, so q is updated 
accordingly. Similarly, pic_residuaL_bits is set to the num- 
ber of bits in the new current picture. 
Allocating the Total Capacity of Medium 207 among the 
Channels: FIGS. 7 and 12 

FIG. 7 shows a flowchart 701 of the CBC control algo- 
rithm that is used to assign the new bitrate for each VBR 
encoder for the next Tc period. Hie control algorithm is a 
loop 713 that executes each T c . At the start of the loop, Rmin 
and Rmax from each TRC(i) are collected. The total avail- 
able bits per Tc parameter, Be, has already been calculated. 
Be will be only updated when there is a change of channel 
bandwidth, Rc, which only happens rarely. Be is calculated 
as 

Bc=Rc*Tc 

where Tc is in units of seconds. 

Be is divided among the bit stream 109 in accordance with 
the ranges of rates specified by the TRCs (0 . . . n) and in 
accordance with a set of priorities which indicate which bit 
streams 109 are more important. The priorities are provided 
by the operator of processor 907 and are set for each bit 
stream when the multiplexer is initialied for the bit stream. 
In the preferred embodiment, there are three levels of 
priority, according to the extent to which timely delivery of 
the pictures in the bit stream is required: ' 

PL=1: Every picture in the bit stream will be delivered, 
and each of them will be delivered on time. 

PL- 2: Some picture will always be delivered on time. For 
example, a picture may be repeated to keep bit buffer 
115(t) fr° m underflowing. 

PL=3: No time guarantees. The bit stream could even be 
interrupted to give the channel to another bit stream. 

PL 1 and 2 are used for real-time video programs. PL 3 is 
used for preemptible data, that is, data which has no real- 
time requirements. Examples of such data are non-real time 
video programs or non-time-dependent data such as E-mail. 
PL 3 permits fuill use of the available bandwidth in situa- 
tions where the sum of the video data is less than the total 
vailablc bandwidth. The total bandwidth available that T c : 
and the priority for each bit stream 109(0 k provided by 
input block 707. The total bandwidth, the priorities, and the 
maximums and minimums for the channels are employed in 
block 705 to allocate a minimal bit rate to each bit stream 
109(0- Details on the algorithm used to do this will be given 
below. 

Once the miinlbit rates for all bit stream 109(0 . . . n) have 
been allocated, the algorithm subtracts the allocated bit rates 
from the total bandwidth to determine whether any band- 
width remains (709). If none is left, the allocation is finished 
and as shown at 711. 721, and 715, the bandwidth allocated 
to each TRC 413(0 * s assigned to it (721) and loop 713 is 
repeated for the next T c . If there are bits left (branch 717), 
the residual bits are assigned to the bit streams 109(0 mat 
can take more bits (719). The algorithm for doing this is also 
explained in more detail below. Once the residua] bits have 
been assigned, blocks 701, 715, and loop 213 are executed 
as described above. There remains, of course, the possibility 
that there is not enough total bandwidth to perform the 
allocation of block 705. This worst-case scenario is called 
Panic mode and will be further discussed later. 
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Minimal bitrate allocation algorithm, FIG. 12 

FIG. 12 shows a flowchart 1201 for this algorithm. The 
algorithm allocates a minimal bitrate to each TRC 413(0 a °d 
returns the number of bits still available to be allocated. The 

5 allocation is ordered by priorities, beginning with PL=1, as 
shown in block 1201. The remainder of the flowchart 
consists of an inner loop 1215, which is executed for each 
TRC 413(0 belonging to a given priority and an outer loop 
1233 which is executed for each priority. The algorithm 
terminates when any of three conditions occurs: 

10 there is no more bandwidth to allocate; 

rates have been allocated to all bit streams 109(0 . . . n); 
allocations have been made for all of the priorities. 
Continuing in more detail with inner loop 1215, in block 
1203, the TRC 413(0 t0 which bandwidth is currently being 

15 allocated receives the amount determined by R min (i) for that 
TRC 413(0- Th c bandwidth is rounded to complete 188-bit 
packets. In decision block 1205, it is determined whether 
there is any bandwidth left. If not, branch 207 is taken, 
terminating loop 1215; if there is, loop 1215 continues to 

20 decision block 1211, where it is determined whether there 
arc more bit streams 109(0 having the current priority. If 
there are, loop 1215 is repeated; otherwise, as indicated by 
branch 1213, the program enters a new iteration of outer 
loop 1213. In that loop, decision block 1215 first checks 

25 whether there is another priority level to be processed; if 
there is (branch 227), PL is incremented and a new set of 
iterations of inner loop 1215 for that priority begins. If there 
is no additional priority level, loop 1233 terminates, as seen 
at branch 1229. 

30 Looking at the termination conditions in more detail, if 
there is no more bandwidth to be allocated, branch 1207 is 
taken. In decision block 1217, it is determined whether there 
are any bit streams 109(0 f° r which a minimal bandwidth 
must still be allocated. If there are none, branch 1219 is 
taken and the remaining bandwidth is returned at 1235. If 

35 there are still bit streams 109(0, the program takes branch 
1221 and enters the panic process 1223, which deals with the 
problem as required by the priorities of bit streams 109(0 . 
. . n) and then returns the remaining bandwidth at 1235. 
Similarly, branch 1229, taken when all priority levels have 

40 been processed, returns the remaining bandwidth at 1235. 
Continuing with panic process 1223, if a bit stream 109(0 
cannot receive the minimum rate it requires, one of two 
things may occur, depending on the bit stream: 
SMB 507(0 mav overflow, causing loss of data. 

45 bit buffer 119(0 in decoder 115(0 mav underflow, causing 
interruption of the display of pictures. 
In the first case, either the input to SMB 507(0 must be 
decreased or the output from bit SMB 507(0 must De 
increased. Generally, the second solution can be employed 

50 in the short term and the first in the longer term. Beginning 
with the second solution, the extra bandwidth must be taken 
from priority 2 and 3 bit streams, beginning with bit streams 
109(0 w * m priority 3. These bit streams have no time 
constraints and can be denied any bandwidth at all for as 

5S long as is necessary. Bandwidth can also be taken from 
priority 2 bit streams 109(0 mat have space in their SMBs 
507(0 by having them output a repeat of a picture until the 
panic condition is over or until their SMB 507(0 threatens 
to overflow. Of course, what the repeat produces at the 
receiver is a still picture. Because the repeal picture is totally 

60 redundant with regard to the picture it is repeating, it always 
has fewer bits than that picture. 

Given that the reason for the substitution is to free up 
bandwidth, it is desirable to make the repeat picture as small 
as possible. That is achieved by sending a repeat of a coded 

65 picture that is not used to predict other pictures. B pictures 
fulfill this criterion, as do P pictures that immediately 
precede an I picture in sequences that do not contain B 
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pictures. The substitution technique requires that transmis- 
sion controller 413 for a PL 2 bit stream respond to an 
indication of a panic from central bitrate controller 1007 by 
reading header information to determine the type and size of 
the picture being output and when it finds the proper kind of 
picture, following it with repeat pictures until the panic is 
over. 

Where the problem is underflow of bit buffer 119, if the 
bit stream is a priority 1 bit stream, extra bandwidth must 
again be found and the techniques described above must be 
applied. If bit stream 109(0 is a priority 2 bit stream, the 
techniques described for priority 1 bit streams may be 
employed, or if that is not possible, the bandwidth required 
for the bit stream may be reduced by outputting a minimal- 
sized repeat picture as described above until the panic 
condition is over or until overflow of SMB 507(0 threatens. 

Where the problem is the threatened overflow of one or 
more SMB buffers 507, it may also be addressed by decreas- 
ing the bit rate at which the encoders 107 produce data. If the 
encoders 107 are co-located with statistical multiplexer 401, 
feedback from multiplexer 401 to the encoders may be used 
to do this. With this kind of feedback, there is no require- 
ment that multiplexer 401 understand the inner workings of 
encoders 107. All that the signal to a given encoder 107(0 
need indicate is that the encoder must reduce its ouput rate 
by some amount. Which encoders receive the signal can be 
determined in many fashions by multiplexer 401. One 
approach is to reduce the bit rate (and therefore the image 
quality) in channels on the basis of their priority levels; 
another is to reduce the bit rate in all channels equally. 
Typically, taking bandwidth from other bit streams would be 
a short-term solution that would be employed until the 
encoding rate could be changed. In the preferred hardware 
embodiment, short-term panic management is done in cen- 
tral bitrate controller 1007, while long-term panic manage- 
ment is done in control processor 907. 
Algorithm for allocating residual bits 

When each of the bit streams 109(0 has received its 
minimum bitrate and there is still bandwidth remaining in 
medium 207, this residual bandwidth B. is allocated among 
the bit streams in the preferred embodiment by allocating 
each bit stream 109(0 an additional bit rate AR(i) which is 
proportional to the difference between the maximum and 
minimum bit rates computed by TRC 413(0 for the bit 
stream. AR(i) is calculated in the preferred embodiment as 
follows: 

_ ^(0-^(0 Be 



In a preferred embodiment, all of the bit rates involved in 
the above computation are rounded to an integer number of 
packets per second. 
Conclusion 

The foregoing Detailed Desrpfion has disclosed to those 
skilled in the arts to which the invention pertains how to 
make and use apparatus for outputting a variable-rate bit 
stream which uses a model of a receiver of the bit stream and 
information from the bit stream to determine a rate at which 
the variable-rate bit stream must be output to avoid overflow 
or underflow in the receiver. The Detailed Description has 
further disclosed a multiplexer which employs the apparatus 
to multiplex a set of variable-rate bit streams onto a medium, 
has disclosed how such apparatus and multiplexers may be 
used with bit streams of video images that are encoded 
according to the MPEG-2 standard, and has given algo- 
rithms for the use of models of MPEG-2 receivers to 
compute rate requirements. 

The Detailed Description has disclosed the best mode 
presently known to the inventors of implementing their 
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apparatus and multiplexers which employ the apparatus; it 
will, however, be immediately apparent to those skilled in 
the arts to which the invention pertains that the invention 
may be employed with variable-rate bit streams other than 
s those which are encoded according to the MPEG-2 standard 
and that many other implementations can be made which 
will incorporate the principles of the invention but employ 
different specific implementation techniques. 

For these reasons, the Detailed Description is to be 
lQ regarded as being in all respects exemplary and not 
restrictive, and the breadth of the invention disclosed herein 
is to be determined not from the Detailed Description, but 
rather from the claims as interpreted with the full breadth 
permitted by the patent laws. 
What is claimed is: 
35 1. A method used in a multiplexer that multiplexes a 
plurality of variable-rate bit streams onto a medium for 
allocating the bandwidth of the medium among the bit 
streams for a period of time, each bit stream having upper 
and lower bounds for the bit rate of the bit stream during the 
20 period of time and the method comprising the steps of: 
for each bit stream, allocating bandwidth from the 
medium as required for the lower bound of the bit rate 
of the bit stream during the period of time; 
determining how much bandwidth remains in the medium 
25 for the period of time; and 

determining the difference between the upper and lower 
bounds for the bit stream and for each bit stream, 
allocating a portion of the remaining bandwidth to 
the bit stream in proportion to the difference, such 
30 that for each bit stream, the bit rate of the bit stream 

during the period of time is no more than the upper 
bound. 

2. The method set forth in claim 1 wherein the step of 
allocating a portion of the remaining bandwidth to the bit 

35 stream includes the step of allocating bit stream i*s portion 
AR(i) according to the formula 

m ~ £(f?jto<0-fc»(0)rc 

i 

40 

where B c is the residual bandwidth for the period of time and 
T c is the period of time. 

3, A multiplexer for multiplexing a plurality of variable- 
45 rate bit streams onto a medium, the multiplexer being 

characterized in that: 

the multiplexer allocates bandwidth belonging to the 
medium to the variable-rate bit streams according to 
any of the methods set forth in claims 1 or 2. 
50 4. Apparatus for providing a plurality of variable-rate bit 
streams to a medium, the apparatus comprising 

a plurality of encoders for receiving constant-rate bit 
streams and producing variable-rate bit streams 
therefrom, 

55 a multiplexer for receiving the variable-rate bit streams 
from the encoders, and 
a transmitter for receiving the variable- rate bit streams 
and outputting the variable-rate bit streams to the 
6Q medium, the apparatus being characterized in that 
the multiplexer is a multiplexer according to claim 1. 
5. The method set forth in any of claims 1 or 2 wherein: 
at least one of the bit streams in the plurality thereof is a 
bit stream produced by encoding a sequence of video 
65 images according to the MPEG-2 standard. 

***** 
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